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METHOD AND APPARATUS FOR GENERATING RECOMMENDATIONS BASED ON 

CONSISTENCY OF SELECTION 

5 Field of the Invention 

The present invention relates to recommendation 
systems, such as recommenders for television programming or other 
content, and more particularly, to a method and apparatus for 
generating recommendations based on the consistency of selections 
10 made by a user. 

Background of the Invention 

The number of media options available to individuals is 
increasing at an exponential pace. As the number of channels 

15 available to television viewers has increased, along with the 
diversity of the programming content available on such channels, 
it has become increasingly challenging for television viewers to 
identify television programs of interest* Historically, 
television viewers identified television programs of interest by 

20 analyzing printed television program guides. Typically, such 
printed television program guides contained grids listing the 
available television programs by time and date, channel and 
title. As the number of television programs has increased, it 
has become increasingly difficult to effectively identify 

25 desirable television programs using such printed guides. 

More recently, television program guides have become 
available in an electronic format, often referred to as 
electronic program guides (EPGs) . Like printed television 
program guides, EPGs contain grids listing the available 

30 television programs by time and date, channel and title. Some 
EPGs, however, allow television viewers to sort or search the 
available television programs in accordance with personalized 
preferences. In addition, EPGs allow for on-screen presentation 
of the available television programs. 
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While EPGs allow viewers to identify desirable programs 
more efficiently than conventional printed guides, they suffer 
from a number of limitations, which if overcome, could further 
enhance the ability of viewers to identify desirable programs. 
5 For example, many viewers have a particular preference towards, 
or bias against, certain categories of programming, such as 
action-based programs or sports programming. Thus, the viewer 
preferences can be applied to the EPG to obtain a set of 
recommended programs that may be of interest to a particular 

10 viewer. t 

Thus, a number of tools have been proposed or suggested 
for recommending television programming. The Tivo™ system, for 
example, commercially available from Tivo, Inc., of Sunnyvale, 
California, allows viewers to rate shows using a "Thumbs Up and 

15 Thumbs Down" feature and thereby indicate programs that the 
viewer likes and dislikes, respectively. Thereafter, the TiVo 
receiver matches the recorded viewer preferences with received 
program data, such as an EPG, to make recommendations tailored to 
each viewer. 

20 Such tools for generating television program 

recommendations provide selections of programs that a viewer 
might like, based on their prior viewing history. Even with the 
aid of such program recommenders , however, it is still difficult 
for a viewer to identify programs of interest from among all the 

25 options. Furthermore, currently available program recommenders 
typically generate a recommendation score based on the user's 
viewing history. Thus, each time a program is watched, the 
positive counts associated with the program are incremented, 
thereafter resulting in a higher program recommendation score for 

30 the program. Currently available program recommenders, however, 
do not consider how frequently the program is watched relative to 
the number of times the program was offered for viewing. 

A need therefore exists for a method and apparatus for 
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recommending content and other items based on the consistency 
with which the user selected the item each time the item was 
offered. 

5 Summary of the Invention 

Generally, a method and apparatus are disclosed for 
generating recommendations for one or more items based on the 
consistency with which an item was selected relative to the 
number of times the item was offered. The present invention 

10 adjusts a conventional program recommender score based on a 
consistency metric. 

The exemplary consistency metric is defined as the 
ratio of the number of times an item was selected over the number 
of times the item was offered in a given time period. Thus, in a 

15 program recommendation implementation, the consistency metric is 
defined as the ratio of the number of times a program was watched 
over the number of times the program was presented in a given 
time period. Thus, generated recommendation scores are increased 
or decreased in an appropriate manner to reward or penalize a 

20 user for consistent or inconsistent, respectively, selection of 
the item. 

A more complete understanding of the present invention, 
as well as further features and advantages of the present 
invention, will be obtained by reference to the following 
25 detailed description and drawings. 

Brief Description of the Drawings 

FIG. 1 illustrates a television programming recommender 
in accordance with the present invention; 
30 ' FIG. 2 is a sample table from the viewer profile 

database of FIG. 1; 

FIG. 3 is a sample table from the program database of 
FIG. 1; and 
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FIG. 4 is a flow chart describing an exemplary program 
recommendation process embodying principles of the present 
invention, 

5 Detailed Description 

FIG. 1 illustrates a television programming recommender 
100 in accordance with the present invention. As shown in FIG. 
1, the television programming recommender 100 evaluates each of 
the programs in an electronic programming guide (EPG) 110 to 

10 identify programs of interest to a particular viewer. The set of 
recommended programs can be presented to the viewer, for example, 
using a set-top terminal /television 180 using well known on- 
screen presentation techniques . 

According to one feature of the present invention, the 

15 television programming recommender 100 generates television 
program recommendations based on the consistency with which a 
given item was selected relative to the number of times the item 
was offered. The present invention adjusts a conventional 
program recommender score based on a consistency metric. The 

20 exemplary consistency metric is defined as the ratio of the 
number of times an item was selected over the number of times the 
item was offered in a given time period. The time period can be 
varied to permit the consistency metric, C m/ to focus on recent 
behavior. The consistency metric may be translated to an 

25 adjustment to the conventional program recommender score, for 
example, using a linear mapping that translates a consistency 
metric, C m , of 0 to a penalty of 25% and a consistency metric, 
C m , of 100 to a reward of 25%. Thus, in the illustrative 
embodiment, the conventional program recommender score can be 

30 increased or decreased by up to twenty five percent (25%) to 
reward or penalize a user for consistent or inconsistent 
selection of the item, respectively. 
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While the present invention is illustrated herein in 
the context of a television program recommender, the present 
invention can be applied to any automatically generated 
recommendations that are based on an evaluation of user behavior, 
5 such as a viewing history or a purchase history. Thus, in a 
program recommendation implementation, the consistency metric is 
defined as the ratio of the number of times a program was watched 
over the number of times the program was presented in a given 
time period. For example, if a given program is presented seven 

10 times each week and the user watched the program five times in a 
given week, then the consistency metric, C m , is 5/7. 

Likewise, in a more general recommendation 
implementation, the consistency metric is defined as the ratio of 
the number of times an item was selected over the number of times 

15 the item was offered in a given time period. For example, the 
item may be books written by a particular author, or a given 
periodical, such as a magazine. 

The television program recommender 100 may be embodied 
as any computing device, such as a personal computer or 

20 workstation, containing a processor 150, such as a central 
processing unit (CPU), and memory 160, such as RAM and ROM. In 
addition, the television programming recommender 100 may be 
embodied as any available television program recommender, such as 
the Tivo™ system, commercially available from Tivo, Inc., of 

25 Sunnyvale, California, or the television program recommenders 
described in United States Patent Application Serial No. 
09/466,406, filed December 17, 1999, entitled ^Method and 
Apparatus for Recommending Television Programming Using Decision 
Trees," (Attorney Docket No. 700772) and United States Patent 

30 Application Serial No. 09/498,271, filed Feb. 4, 2000, entitled 
*Bayesian TV Show Recommender," (Attorney Docket No. 700690), or 
any combination thereof, as modified herein to carry out the 
features and functions of the present invention. 
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As shown in FIG* 1, and discussed further below in 
conjunction with FIGS. 2 through 4, respectively, the memory 160 
of the television programming recommender 100 includes one or 
more viewer profile (s) 200 r a program database 300 and a program 
5 recommendation process 500. Generally, the illustrative viewer 
profile 200 provides feature counts derived from the user's 
viewing history. The program database 3 00 records information 
for each program that is available in a given time interval. 
Finally, the program recommendation process 400 generates 

10 recommendation scores for each program in a particular time 
interval, taking into account the consistency with which a given 
program was selected relative to the number of times the program 
was presented. 

FIG. 2 is a table illustrating an exemplary implicit 

15 viewer profile 200. As shown in FIG. 2, the implicit viewer 
profile 200 contains a plurality of records 205-213 each 
associated with a different program feature. In addition, for 
each feature set forth in column 23 0, the implicit viewer profile 
200 provides corresponding positive counts in fields 23 5 and 

20 negative counts in field 250. The positive counts indicate the 
number of times the viewer watched programs having each feature. 
The negative counts indicate the number of times the viewer did 
not watch programs having each feature. 

For each positive and negative program example (i.e., 

25 programs watched and not watched) , a number of program features 
are classified in the user profile 200. For example, if a given 
viewer watched a given sports program ten times on Channel 2 in 
the late afternoon, then the positive counts associated with 
these features in the implicit viewer profile 200 would be 

30 incremented by 10 in field 235, and the negative counts would be 
0 (zero) . Since the implicit viewing profile 200 is based on the 
user's viewing history, the data contained in the profile 2 00 is 
revised over time, as the viewing history grows. Alternatively, 
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the implicit viewer profile 200 can be based on a generic or 
predefined profile, for example, selected for the user based on 
his or her demographics . 

Although the viewer profile 2 00 is illustrated using an 
5 implicit viewer profile, the viewer profile 200 may also be 
embodied using an explicit profile, or a combination of explicit 
and implicit profiles, as would be apparent to a person of 
ordinary skill in the art. For a discussion of a television 
program recommender 100 that employs both implicit and explicit 

10 profiles to, obtain a combined program recommendation score, see, 
for example, United States Patent Application Serial Number 
09/666,401, filed September 20, 2000, entitled "Method And 
Apparatus For Generating Recommendation Scores Using Implicit And 
Explicit Viewing Preferences," (Attorney Docket Number 701247), 

15 incorporated by reference herein. 

FIG. 3 is a sample table from the program database 3 00 
of FIG. 1 that records information for each program that is 
available in a given time interval. The data that appears in the 
program database 300 may be obtained, for example, from the 

20 electronic program guide 110. As shown in FIG. 3, the program 
database 300 contains a plurality of records, such as records 305 
through 320, each associated with a given program. For each 
program, the program database 3 00 indicates the date/ time and 
channel associated with the program in fields 340 and 345, 

25 respectively. In addition, the title and genre for each program 
are identified in fields 350 and 355. Additional well-known 
attributes (not shown) , such as actors, duration, and description 
of the program, can also be included in the program database 300. 

The program database 300 may also optionally record an 

30 indication of the recommendation score (R) assigned to each 
program by the television programming recommender 100 in field 
370. In addition, the program database 300 may also optionally 
indicate in field 370 the adjusted recommendation score (A) 
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assigned to each program by the television programming 
recommender 100 in accordance with the present invention. In 
this manner, the numerical scores, as adjusted by the present 
invention, can be displayed to the user in the electronic program 
5 guide with each program directly or mapped onto a color spectrum 
or another visual cue that permits the user to quickly locate 
programs of interest. 

FIG. 4 is a flow chart describing an exemplary program 
recommendation process 400 embodying principles of the present 

10 invention. As shown in FIG. 4, the program recommendation 
process 400 initially obtains the electronic program guide (EPG) 
110 during step 410. Thereafter, the program recommendation 
process 400 calculates the program recommendation score, R, 
during step 420 for each program in the time period of interest 

15 in a conventional manner (or obtains the program recommendation 
score, R, from a conventional recommender) . 

Thereafter, the program recommendation process 400 
calculates the consistency metric, C M / for each program in the 
time period of interest during step 430. A test is then 

20 optionally performed during step 440 to determine if the 
calculated consistency metric, C m , is below a predefined 
threshold. Generally, the test performed during step 440 is 
intended to prevent a user from being penalized if the viewer 
failed to watch a program at all, or only watched the program a 

25 nominal amount of times. 

If it is determined during step 440 that the calculated 
consistency metric, C m , is below a predefined threshold, then the 
consistency metric, Cm, is calculated during step 450 for similar 
programs that may be relevant to the consistency of the current 

30 program. Generally, similar program may be identified, for 
example, by evaluating a closeness metric that compares the 
various program features of two programs. The similarity can be 
computed, for example, as a dot product of two feature vectors 
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corresponding to the television programs. Generally, if SI and 
S2 are the two shows, the shows can be characterized as: Si 
(Program 101) : < genre: comedy, genre: situation, genre: family, 
channel: NCB> and S2 (Program 228): <genre: comedy, genre: 
5 situation, genre: family, channel: NCB>. The dot product of SI 
and S2 would be a weighted, normalized average. A weight can be 
assigned for the similarity of each feature, such as genre and 
channel similarities. In addition, certain features, such as 
day-time may be optionally considered in the computation because 

10 if the, shows are on the same channel, two programs will never be 
on the same channel at the same time. Using the day-time feature 
makes sense only in the case of different channels. The weights 
should add up to 1.0. 

If, however, it is determined during step 440 that the 

15 calculated consistency metric, C m/ is not below a predefined 
threshold, then the calculated consistency metrics, C m , for each 
program in the time period of interest (or for similar programs 
if the consistency metric, C m , was below a threshold) are 
translated during step 460 to an adjustment factor, F, for 

20 example, using a linear mapping, and then the adjusted program 
recommendation score, A, is also calculated during step 460 for 
each program in the time period of interest, as follows: 
A = R#F. 

The program recommendation process 400 then calculates 
25 the combined program recommendation score, C, during step 470 for 
each program in the time period of interest, as follows: 
C = MIN{A, 100} 

Thus, the exemplary program recommendation process 400 ensures 
during step 470 that the combined program recommendation score, 
30 C, does not exceed 100% (the maximum score) . 

Finally, the program recommendation process 400 
provides the combined program recommendation scores (C) for the 
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programs in the time period of interest to the user during step 
450, before program control terminates. 

In further variations of the program recommendation 
process 400, the adjusted program recommendation score, A, may be 
5 calculated during step 43 0 using a bonus scoring system, wherein 
a predefined or fixed bonus is determined, for example, based on 
the consistency metric. 

It is to be understood that the embodiments and 
variations shown and described herein are merely illustrative of 
10 the principles of this invention and that various modifications 
may be implemented by those skilled in the art without departing 
from the scope and spirit of the invention. 
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